• x
p(x)
•
˜
x
x
x
• x
x
x
• p(x)
32×32 2
3072
10
800
x
n k
P (x)
k
n
•
n k
•
32 ×32
•
P (x)
P (
1
)
P (
2
|
1
)
•
u ∼ U(0, 1)
u
t
0
t
1
t
2
Alice Bob Carol
0 1
1 2
0 1
2 1 0 2
1 0
x
G
p(
i
| P a
G
(
i
)) P a
G
(
i
)
i
G x
p(x) = Π
i
p(
i
| P a
G
(
i
)).
p(
0
,
1
,
2
) = p(
0
)p(
1
|
0
)p(
2
|
1
).
0 1 2
p(
0
,
1
,
2
)
0
×
1
×
2
0 1 0
2 1
n k
O(k
n
)
m
O(k
m
) m << n
O(k)
O(k
2
)
0 1
1
0 1
0 1
2
k × k −1
0 1
k − 1
h
r
h
y
h
c
r
y c
y
r
c
G C φ(C)
˜p(x) = Π
C∈G
φ(C).
y c
y
= 0
y
= 1
c
= 0
c
= 1
y r
p(x) =
1
Z
˜p(x)
Z
Z =
˜p(x)dx.
Z φ
φ Z
Z
Z
Z
x
φ Z
Z
Z
˜p
∈ R
φ(x) = x
2
Z =
x
2
dx.
φ(x) φ
φ(x; β) = exp
−βx
2
β Z
β β φ
φ
φ
n
x
b x φ
i
(
i
) = exp(b
i i
)
x
x ∈ R
n
Z
x ∈ {0, 1}
n
p(x) n
p(
i
= 1) = (b
i
) x
{[1, 0, . . . , 0], [0, 1, . . . , 0], . . . , [0, 0, . . . , 1]} p(x) = (b)
b
i
p(
j
= 1) j = i
φ
∀x, ˜p(x) > 0
a b c
d e f
p(A, B, C, D, E, F)
1
Z
φ
A,B
(A, B)φ
B,C
(B, C)φ
A,D
(A, D)φ
B,E
(B, E)φ
E,F
(E, F )
φ
˜p(x) = exp(−E(x))
E(x) exp(z)
z
x
exp(a) exp(b) = exp(a + b)
− −
E E
−
Z
a b c
d e f
E( , , , , , ) E
,
( , ) +
E
,
( , ) + E
,
( , ) + E
,
( , ) + E
,
( , )
φ φ
exp φ
,
( , ) = exp (−E( , ))
E
A
B S A
B S
a s b
a s b
ra
a
b c
d
A
B S
A B S
a s b
a s b
a s b
a
s
b
a s b
c
•
•
•
•
•
= 1
= 0
D
U
U
D D U
D
U U
U
U
U
D
D
D
h
1
h
2
h
3
v
1
v
2
v
3
a b
c
a cb
h
1
h
2
h
3
v
1
v
2
v
3
a b
c
a cb
⊥
D U
φ
φ
φ
φ
v
h v
h
v
h
v
v
p( )
v h
E[h | v]
h
p(h, v) v
h
x
h h
h
h h
h ∼ p(h | θ)
h = f(θ, η)
η
p(η) f
∂
∂θ
L(h)p(h | θ)dh
θ h
L
η
h
∂
∂θ
L(f(θ, η))p(η)dη.
h
g =
∂L(f(θ, η))
∂θ
p(η)dη.
ˆg =
∂L(f(θ, η))
∂θ
η ∼ p(η) E[ˆg] = g
h
1
h
2
h
3
v
1
v
2
v
3
h
4
E(v, h) = −b
v −c
h − v
W h
b c W
p(h | v) = Π
i
p(
i
| v)
p(v | h) = Π
i
p(
i
| h).
p(
i
= 1 | v) = σ
v
W
:,i
+ b
i
.
h v
∂
∂W
i,j
E
v,h
E(v, h) = −
i j
.
∇
θ
log Z